Worst-case complexity and empirical evaluation of artificial intelligence methods for unsupervised word sense disambiguation

Word Sense Disambiguation (WSD) is a difficult problem for NLP. Algorithm that aim to solve the problem focus on the quality of the disambiguation alone and require considerable computational time. In this article we focus on the study of three unsupervised stochastic algorithms for WSD: a Genetic Algorithm (GA) and a Simulated Annealing algorithm (SA) from the state of the art and our own Ant Colony Algorithm (ACA). The comparison is made both in terms of the worst case computational complexity and of the empirical performance of the algorithms in terms of F1 scores, execution time and evaluation of the semantic relatedness measure. We find that the worst-case complexity of GA is a factor of 100 higher that SA. However, it is difficult to make any comparison to ACA. We estimate the best parameters manually for SA and GA, but automatically for ACA (made possible by its short execution time). We find that ACA leads to a shorter execution time (factor of 10 and 100, respectively) as well as better results. Using different voting strategies, we find a small increase in the F1 scores of SA and GA and significant improvements in the results of the ACA. With the latter, we surpass the First Sense baseline and come close to the results of supervised systems on the coarse-grained all words task from Semeval 2007. Copyright c © 2009 Inderscience Enterprises Ltd. 2 D. Schwab, J. Goulian and A. Tchechmedjiev